# INT4 Quantization

This is an INT4-quantized Mistral-Small-3.1-24B-Instruct-2503 model, optimized and released by Red Hat (Neural Magic), suitable for fast-response dialogue agents and low-latency inference scenarios.

INT4 quantized version based on the gemma-3-4b-it model, significantly reducing storage and computational resource requirements

This is the quantized version of openai/whisper-large-v3, employing INT4 weight quantization and FP16 activation quantization, suitable for vLLM inference.

INT4 quantized version of FLUX.1-Depth-dev, capable of generating images from text descriptions while adhering to the structure of the input image. Compared to the original BF16 model, this version saves approximately 4x memory and improves runtime speed by 2-3x.

FLUX.1-dev is a text-to-image generation model quantized to INT4 format using Optimum Quanto, suitable for non-commercial use.

A quantized version of Meta-Llama-3.1-8B-Instruct, optimized to reduce disk space and GPU memory requirements, suitable for chat assistant scenarios in English business and research.

INT4 quantized version of Llama 3.1 70B Instruct, optimized with AutoAWQ technology, suitable for multilingual dialogue scenarios.

INT4 quantized version of Llama 3.1 8B Instruct, quantized using AutoAWQ tool, suitable for multilingual dialogue scenarios.

Whisper is a pre-trained model for automatic speech recognition (ASR) and speech translation. This repository provides the Whisper large model in ONNX format with INT4 weight quantization, powered by Intel® Neural Compressor and Intel® Transformers Extension.